Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-4771][Docs] Document standalone cluster supervise mode #3627

Closed
wants to merge 2 commits into from
Closed

[SPARK-4771][Docs] Document standalone cluster supervise mode #3627

wants to merge 2 commits into from

Conversation

andrewor14
Copy link
Contributor

@tdas looks like streaming already refers to the supervise mode. The link from there is broken though.

andrewor14
andrewor14 reviewed Dec 5, 2014
View reviewed changes
docs/spark-standalone.md
@@ -257,7 +257,7 @@ To run an interactive Spark shell against the cluster, run the following command

You can also pass an option `--total-executor-cores <numCores>` to control the number of cores that spark-shell uses on the cluster.

# Launching Compiled Spark Applications
# Launching Spark Applications
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked that there are no references to this section

@SparkQA
Copy link

SparkQA commented Dec 5, 2014

Test build #24197 has started for PR 3627 at commit 2b55ed2.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Dec 6, 2014

Test build #24197 has finished for PR 3627 at commit 2b55ed2.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24197/
Test PASSed.

tdas
tdas reviewed Dec 9, 2014
View reviewed changes
docs/spark-standalone.md
@@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. `
To control the application's configuration or execution environment, see
[Spark Configuration](configuration.html).

Additionally, standalone `cluster` mode supports restarting your application on failure. To use
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you define what is meant by failure? How is failure detected? Non-zero exit code?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, the cluster mode can be optionally asked to automatically restart your application if it fails with non-zero exit code.

@SparkQA
Copy link

SparkQA commented Dec 10, 2014

Test build #24284 has started for PR 3627 at commit 9ca0908.

  • This patch merges cleanly.

tdas
tdas reviewed Dec 10, 2014
View reviewed changes
docs/spark-standalone.md
@@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. `
To control the application's configuration or execution environment, see
[Spark Configuration](configuration.html).

Additionally, standalone `cluster` mode supports restarting your application automatically if it
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: my point was also that it is known that we are talking about "standalone" so you can just say "cluster mode". But its okay.

@tdas
Copy link
Contributor

tdas commented Dec 10, 2014

LGTM.

@SparkQA
Copy link

SparkQA commented Dec 10, 2014

Test build #24284 has finished for PR 3627 at commit 9ca0908.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24284/
Test PASSed.

@andrewor14
Copy link
Contributor Author

Ok merging into master thanks. I'll add a note to back port this later.

@asfgit asfgit closed this in 5621283 Dec 10, 2014
asfgit pushed a commit that referenced this pull request Dec 10, 2014
[SPARK-4771][Docs] Document standalone cluster supervise mode
1da1937 
tdas looks like streaming already refers to the supervise mode. The link from there is broken though.

Author: Andrew Or <andrew@databricks.com>

Closes #3627 from andrewor14/document-supervise and squashes the following commits:

9ca0908 [Andrew Or] Wording changes
2b55ed2 [Andrew Or] Document standalone cluster supervise mode
@andrewor14 andrewor14 deleted the document-supervise branch December 10, 2014 20:43
@andrewor14
Copy link
Contributor Author

JK I just back ported this into branch-1.2 and 1.1.

asfgit pushed a commit that referenced this pull request Dec 10, 2014
[SPARK-4771][Docs] Document standalone cluster supervise mode
273f2c8 
tdas looks like streaming already refers to the supervise mode. The link from there is broken though.

Author: Andrew Or <andrew@databricks.com>

Closes #3627 from andrewor14/document-supervise and squashes the following commits:

9ca0908 [Andrew Or] Wording changes
2b55ed2 [Andrew Or] Document standalone cluster supervise mode
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@andrewor14 @SparkQA @AmplabJenkins @tdas